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A research and development program prioritization 
process proposed by the Marine Corps Development and 
Education Command is investigated. Relevant literature is 
reviewed and the prcoess classified with the majority of 
accepted industrial program selection models. Linear 
progr aiming formulations of the process illustrate resource 
allocation improvements suggested by the literature. Two 
linear programming approaches are demonstrated with avail- 
able process test data. The subjective research and devel- 
opment program values proposed by the original process are 
discussed in terms of measurement scale properties, and 
further research is suggested in the areas of alternative 
model forms, subjective program evaluations, and model 
implementation cost-benefit analysis. 
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I. INIEODOCTION 



This thesis concentrates on the Marine Corps Development 
and Education Ccmmand, Research and Development Program 
Prioritization Process. The process was originally proposed 
to assist in developing the Marine Corps' input to the 
Navy's Research, Development, Test and Evaluation (FDTSE,N) 
Program Ctgective Memorandum (POM) . The process comtines 
subjective evaluations of the Marine Corps' identified 
mission deficiencies and proposed EDT&E programs, and 
produces a RDTSE program rank ordering according to mission 
deficiency accomplishment importance. The Department of 
Defense acquisition system is reviewed as it establishes the 
base for Marine Corps RDTSE program acquisition. Review of 
current acquisition procedures reveal that analytic models 
are used in support of the Procurement Marine Corps (PMC) 
POM but not for Marine Corps RDTSE, N POM input. 

The literature concerning research and development 
program selection models is reviewed. The review shows that 
a wide range of program selection models are proposed and 
accepted by industry. The literature lends credence to the 
MCDEC process by allowing process classification with the 
majority of industry accepted models. Mathematical program- 
ming enhancement to this classification is suggested by the 
literature and is illustrated in this investigation. Linear 
programming formulations are used to duplicate the MCDEC 
process and suggest mathematical programming advantages 
under budget reduction conditions. A MCDEC produced process 
test data set is used to further illustrate the linear 
programming formulations. A second linear programming 
approach is suggested which concentrates on the number of 
mission deficiencies accomplished rather than the RDTSE 
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II. eackge^nd 



A. IBIECDDCTION 

Ihe Flanning/ P i eg ramming , and Budgeting System (PPES) 
structures the defense systems acjuisition procedure within 
the Eepartment of Defense (DoD) . This structure is a step- 
wise review and decision process conducted at various levels 
in the EoD hierarchy. The process translates validated 
defense mission needs into budget allocations which fund 
programs that are expected to overcome the stated needs. In 
this thesis the hierarchical level of concern descends from 
the Office cf the Secretary of Defense (OSD)/ through the 
Department of the Navy (DoN) / to the Marine Corps and the 
Navy. The analysis concentrates on the initial PPBS steps 
taken at this level, and specifically the Research, 
Development, Test, ^and Evaluation (EDTSE) acquisition 
program pricriti zaticr and funding profile process used by 
the Marine Corps. 

E. SYSTEMS ACQOISITICN IN OSMC 

1 • Resronsibilit ies 

Specific responsibilities are assigned the Marine 
Corps by law.i Figure 2.1 depicts three levels of RETSE and 
systems acquisition management that assist in accomplishing 
these responsibilities. The RDT&E and system acquisition 
responsibilities stated in MCO 5000. 10A [Ref. 1] first 

iThe National Security Act of 1947, as amended, and 
subsequent DoD and SecNav directives assign the Marine Corps 
general and specific responsibilities for developing the 
tactics, techniques, and equipment used by amphibious 
landing forces. The Marine Corps is directed to coordinate 
and request appropriate assistance for this development with 
the Navy and other military services. 



desiguates the Assistant Commandant of the Harine Corps 
(ACMC) as the acguisition executive and then assigns him the 
responsibility for overall integration and unification of 
the management process. Ihe Deputy Chief of Staff (CC/S) 
for Eesearch, Development, and Studies (RDSS) assists the 
ACMC by acting as the coordinator of staff activities 
involved in RDTSE and system acguisition from program initi- 
ation to approval for service use. The Commanding General, 
Marine Corps Development and Education Command (CG, MCDEC) 
is the field representative of the Commandant of the Marine 
Corps (CMC) in RDT6E efforts and is responsible for the 
field execution and coordination of Marine Corps' RDT5E 
activities to support the systems acguisition. This 
includes coordinating Marine Corps support needs with other 
military services, as well as rep^'orting on RDT&E efforts of 
other services, agencies, and governments. 




Figure 2. 1 RDISE and System Acguisition levels. 

Ihe deputy chiefs of staff or directors of major 
staff offices at Headguarters Marine Corps (HQMC) are 
assigned as program sponsors for specific Marine Corps 
mission areas and are intimately involved in the acguisition 
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process. The program sponsors are responsible for the 
continuous analysis of their mission areas and the overall 
planning, coordination, and direction of related program 
acguisiticns. These responsibilities are accomplished in 
coordination with DC/S RDSS and CG, MCDEC. 

2 • Ana lys is of N ee ds 

The directed continual analysis of mission areas by 
the program sponsors, with input from the Fleet Marine Force 
(FMF) , initially identifies operational mission needs. 
These needs, or eguivalently mission deficiencies, are 
formally refined, additional needs identified, and related 
new concepts formulated by Mission Area Analysis (MAA) . 
HAAS are assessments of current or projected Marine Corps 
capabilities within the specific mission areas and elements 
listed in Table I and are conducted by HQMC or MCDEC staff 
sections in coordination with the program sponsors and CG, 
MCDEC. The MAA details the mission elements of Table I into 
general operational missions and then further into specific 
operational missions. These specific missions can then be 
matched against specific resources for their accomplishment. 
If no resources are available then a deficiency is identi- 
fied and recommendations for overcoming the deficiency are 
presented. SECNAVINST 5000. 1A [Ref. 2] and "A Guide for the 
Eerfcrmance of USMC Mission Area Analysis'* [Ref. 3] detail 
the reguirements and methodology of conducting MAAs. Each 
identified deficiency is formulated into a Justification for 
Major System New Start (JMSNS) or a Required Operational 
Capability (ROC) according to the Acquisition Category 
(ACAT) . This documentation then forms the basis for initi- 
ating RDTSZ funding and program acquisition. 
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AREA 

210 

220 

230 

2£)0 

250 

260 

320 

430 

450 

460 

470 



TABLE I 

arine Corps Mission Areas and Elements 



ELEMENT 



TITLE 



21 1 
212 

213 

214 

215 

216 



221 

222 

224 

225 



235 

243 



Land Warfare 

Close Ccmhat 

Fire Support 

Ground Air Defense 

Mine Warfare 

Land Combat Support 

Land Combat Service Support 

Air Warfare 
Counter Air 
Offensive Air 
Defense Suppression 
Air Warfare Support 

Naval Warfare 
Amphibious Warfare 

Tactical Nuclear Warfare 
Defensive Tactical N-Karfare 



252 

255 

256 

257 



Theater and Tactical C^I 
Tactical Command Control 
and Intelligence Systems 
Surveillance, Reconnaissance S 
Targets for Acquisition 
Tactical Communications 
Electronic Warfare & Counter C^I 



26 1 


Mobi lity 
Airlift 


262 


Sealift 


264 


Refu eling 


32 1 


Defense Wide 
Navigation S 


322 


Support and 


324 


COM 5 EC 



C^i Support 
Position Fixing 
Base Communication 



Non- System Training Devices 
431 Training Devices/Simulators 

Test and Evaluation Support 

453 Joint Test and Evaluation 

454 Other Test and Evaluation 



46 1 

47 1 



521 

522 

523 



International Cooperative RDTSE 
Standardization & Interoperability 

Management Support 
General Management Support 

Exploratory Development 
Electronic 5 Physical Sciences 
Environmental 5 Life Sciences 
Engineering Technology 



520 



C. EEVIEW fiND DECISION PROCESS 

1 . Kile sto n es 

Four formal decision points are established within 
the program acquisition process and are designated mile- 
stones 0, I, 11, and III. Positive approval is necessary at 
each iiilestcne for passage of a program from conception to 
operational deployment. Milestone 0 signifies program initi- 
ation cn approval of a JMSNS or a ROC. A favorable mile- 
stone I decision gives the approval to demonstrate selected 
alternatives. Then a milestone II approval authorizes full 
scale development and limited production for operational 
testing and evaluation. Finally, milestone III passage is 
necessary for production release and approval for service 
use. Marine Corps' input to the RDTSS, N Program Objective 
Memorandum (POM) is required throughout this process until 
milestone III, after which acquisition funding comes under 
the Procurement Marine Corps (PMC) POM. 

2 . Pha ses 

Ihe review steps of the acquisition process, shewn 
in Figure 2.2 , are described as phases separated by the 
decision milestones. The initial reviews and analysis 
previously discussed are included in the first, or research 
phase. Studies and MAAs conducted in this phase identify 
mission deficiencies that, in turn, generate JMSNSs or FOCs. 
These documents assess the projected threat, state the 
mission element deficiency, identify the existing DCD capa- 
bilities, assess the impact of not acquiring or maintaining 
the capabilities, and finally provide a program plan to 
identify and explore competitive alternatives. 

Approval of the JMSNS or ROC at milestone 0 moves 
the process from the research phase to the program initia- 
tion phase. The objectives of this phase include further 
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Development 
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Development 
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Evaluation 
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Production 
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Production 
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Figure 2.2 Marine Corps Acquisition Process Flow Chart. 
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program study and the development of a technological tase to 
support the program through the remaining reviews and deci- 
sions. lasks necessary to accomplish these objectives are 
program cost analysis, operational effectiveness studies, 
alternative identification, technical and economical feasi- 
bility studies, risk determination, and further concept and 
threat analysis. The primary documents produced during this 
phase are the System Concept Paper (SC?) for major programs 
or the Decision Coordinating Paper (DCP) for less-than-ma jor 
programs. These documents summarize the program development 
to date and address mission element need, technology assess- 
ment, program description, management plan, and acquisition 
logistics and manpower. 

Approval of the SCP or DC? at milestone I starts the 
demonstration and validation phase. Viable alternative 
systems and critical subsystems are subjected to competitive 
demonstrations during this phase. Tasks accomplished in 
conjucticn with this phase include reviews of acquisition 
strategies, logistics, manpower and training planning, as 
well as the preparation of the test and evaluation master 
plan (TEMP). Major consideration is given to the thorough 
understanding of the operationa‘1 need and the evaluation of 
all alternatives. Each alternative's unit cost goal, life 
cycle cost, technical feasibility, and economic realism is 
reviewed. When the demonstration and validation phase 
produces sufficient evidence that the preferred system can 
fulfill all necessary capabilities, and that technology 
exists tc produce the system, the program sponsor recomicends 
approval at milestone II. 

A successfully passed milestone II initiates the 
full scale development (FSD) phase. This phase ensures that 
the engineering and production design, test and evaluation, 
personnel and training, and the integrated logistic support 
planning are completed prior to moving into production. Upon 
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completicn of all research ard development tn rough this 
phase^ the approval for service use is given at milestone 
III. This decision to proceed starts the production and 
deployment phase and also designates the shifts of funding 
from EDTSE/N to PMC as well as the coordination responsi- 
bility from the DC/S EDSS to the Deputy Chief of Staff for 
Installations and Logistics. 

3 • Ca te gories 

All mission-essent ial acquisition programs in the 
Marine Corps are classified as either a major or less-than- 
major program. These two categories imply the level at 
which a milestone decision is made. Major programs are 
designated according to their funding level and ether 
criteria/ and may he assigned by the Secretary of Defense 
(SecDef) / Secretary of the Navy (SecNav) , or the Commandant 
of the Marine Corps (CMC). Department of Defense 
Instruction 5000.2 [Eef. 4]/ SECNAVINST 5000. 1A [Eef. 2], 
and MCO 5000.1 OA [Ref. 1] list major program criteria used 
in these designations. Less-than-ma jor programs are simply 
all other miss ion- essential acquisition programs not desig- 
nated as major programs. 

^ • Eev iew C ou nc ils 

The level of hierarchy that designates a program as 
major also becomes the final decision authority for that 
program at milestones 1/ II , and III. The decision makers 
are supported by recommendations developed by the appro- 
priate review councils which convene for these three mile- 
stones. The Defense System Acquisition Review Council 
(DSAEC) and the Department of the Navy System Acquisition 
EevieW' Council (DNSAEC) provide program recommendations to 
the SecDef and the SecNaV/ respectively. Similarly/ the 
Marine Corps System Acquisition Review Council (MSAEC) 
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supports the CMC or all major programs including these 
subject to higher review. The MSAEC consists of a hoard of 
general officers, which is chaired by the ACMC and includes 
the CG, MCCEC, DC/S E£5S, and other major HQMC staff chiefs 
and directors. This council receives a comprehensive review 
cf the research and development concerning the acquisition 
program and formulates recommendations for submission tc the 
CMC. 

The less-than-major programs undergo a similar 
review process by an In Process Beview (IPS) Committee. This 
committee is chaired by the DC/S EDSS and has representation 
from all major HQMC staff and the CG, MCDEC. The ACMC is the 
decision authority fer less-than-major programs, and the IPR 
committee submits recemmendatiens to the ACMC for the acqui- 
sition program decision. 

E. OSMC BDTSE PEOGBABMING AND BUDGETING 

All research, development, testing, and analysis 
described in this chapter depend critically on 
Congressionally-approved allocations. The estimation and 
submission cf funding requests are coordinated by the DC/S 
ED&S for each fiscal year, and this process involves all 
Chiefs cf Staff and Directors at HQMC. Initially each 
program sponsor must submit a prioritized listing and accu- 
rate funding estimate of his respective acquisition 
programs. The programs are then combined and subjectively 
prioritized and evaluated through a sequence of committees. 
The program sponsors present each proposal before a Program 
Evaluation Group (PEG) and then a POM Working Group (PWG) . 
The groups’ results are then reviewed and evaluated prior to 
submission to the CMC for approval. The approved acquisition 
program funding profiles levels are included in the Navy 
portion of the Five lear Defense Plan (FYDP) and become the 
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Marice Corps’ input to the RDTSE,N POJl. After DoN, EOD, and 
Congressional negotiations, Marine Corps EDT&E funds are 
appropriated by Congress as part of the RDT&E,N budget. 

A siirilar cyclic process is involved in the development 
and submission of the PMC PCM for the allocation of funds 
for the production and deployment phase after a positive 
milestone III decision. One difference between the two 
processes is the existence of an analytic support model used 
in the EMC process and the current absence of a support 
model for the RDT5E process. The model used during the PMC 
POM process was developed under civilian contract and is 
described in the final contract report, "Decision-Analytic 
Support of the USMC Program Development: A Guide to the 

Methodology" [Ref. 5]. Independent of the PMC model, the 
CG, MCEEC developed a process with the objective of combining 
many subjective judgements from headquarters and operational 
levels to prioritize RDTSE acquisition programs and provide 
suggestions in program funding profiles, however, this 
process has not been implemented. 

E. TEE PROELEM 

Defense system research and development programs indi- 
vidually do not possess an obvious numerical quantity that 
allows mathematically straightforward program comparisons or 
funding profile optimization. Inherent then to RDTSE 
program and funding decisions are the personal judgments of 
the decision maker. These subjective decisions rely on the 
information and analytic support available to the individual 
and supporting staff group prior to the decision. Marine 
Corps RDTSE acquisition programs are prioritized in upper 
hierarchical level committees and currently do not utilize 
mathematical analytical decision support. The increasing 
fiscal commitment associated with each new defense program 
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III. IITEMIORE REVIEW 



A. IMECDDCTION 

The iirpcrtance cf Research and Development (RSD) in the 
private and governmental sectors is evident from the quan- 
tity cf information available in professional papers^ jour- 
nals, and books. Ihis chapter reviews a portion of this 
literature which concentrates on RSD project selection and 
resource allocation models, the suggested utilization of 
these models, their limitations, and their acceptability. 
Ihe purpose of this review is to investigate demonstrated or 
proposed methods of solving a problem similar to that previ- 
ously presented and should not be viewed as an exhaustive 
study of RSD literature. 

Several published articles review a number of proposed 
methodologies and models in the RSD project selection and 
resource allocation area. These review articles tend to 
classify model forms into one of two general categories. 
Baker [Ref. 6] titled the categories as "benefit measure- 
ment" and "project selection and resource allocation." 
Winkcfsky, et al. [Ref. 7] described similar classifica- 
tions as "value measurement" and "portfolio selection." 
Benefit or value measurement models rely on subjective 
expert judgement for project evaluation combined into check- 
lists, comparative project scores, or economic indices. 
Project or portfolio selection and resource allocation 
models normally use a mathematical programming or other 
optimizing technique to maximize a benefit, such as profit, 
subject to constraints, such as budget and manpower. ether 
articles, Augood [Ref. 8], Gear, et al. [Ref. 9], Gear 
[Ref. 10], Newman [Ref. 11], and other authors review or 
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propose lodels that can be classified into these general 
categories. Benefit measurement and portfolio selection will 
be used for model cl assific aticn in this review. 

Some authors, Winkofsky, et al. [Ref. 12], 

Baker, et al. [Ref. 13], and Allen, et al. [Ref. 14], 

propose methodologies for implementing series of models for 

hierarchical applications. In addition to the model and 
methodology proposal and review articles, Brandenburg 

[Ref. 15], Baker [Ref. 6], Souder [Ref. 16 and 17], and 
other authors suggest R5D model analysis criteria as well as 
observed model limitations and acceptability. 

E. RESEARCH AND DEVEIOPMENT MODELS 

In addition to the general classifications of benefit 
measurement and portfolio selection, models are further 
defined by a common form, such as an index model or a linear 
programming model, a description of that form, and the 
recommended or observed use of the model. 

1 • Ws^surem^t Mo de ls 

Benefit measurement models typically use subjective 
evaluation of a number of R5D project attributes and combine 
these evaluations into a project value or worth. Ihese 
models are classified into the checklist, scoring and index 
model forms. 

a. Checklist Models 

These models are the simplest type and use 
subjective comparisons of a project against a list of 
elements. The elements describe project criteria or 
considerations, established by the management, that have 
some significance towards success. Augood [Ref. 8] suggests 
five general criteria subdivided into 53 specific elements. 
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Cooper [Eef. 18] surveyed 103 industrial product firms and 
was atle tc analytically reduce his 48 similar elements into 
13 meaningftl factors. Augood [Hef. 8] also suggests rating 
projects against the checklist elements by using descriptors 
such as very unfavorable, unfavorable, average, favorable, 
and very favorable. The worth of a project is provided by 
observing the number of criteria it meets or how "favorable" 
the project looks by its overall element descriptor ratings. 
This method then requires judgemental evaluations of the 
checklist results fcr each project. Winkofsky, et al. 
[Ref. 7] in their literature review, observed that checklist 
models are most suitable for project evaluation in the 
exploratory ESD phase when only preliminary project informa- 
tion is available. 

b. Scoring Models 

Scoring models are quantified extensions of the 
checklist models and provide a numerical score or value fcr 
each project. Dsing a similar criteria list as used in the 
checklist models, the project score is typically generated 
from picducts of assigned criteria weights and project 
values associated with criteria accomplishment. These prod- 
ucts are then summed over all criteria to provide the final 
project score. Winkcfsky, et al. [Bef. 7] presented the 
general scoring model form as 

PVj = Sum/i [ (Wi) (Sij) ] (3.1) 



where , 
i = 

j 

PVj = 
Wi = 
Sij = 



criterion number, 
project number, 

jth project value to organization, 

ith criteria or element weight, and 

jth project accomplishment of the ith criterion. 



24 



The weights such as Ki or 
iranagers or E6D advisory g 
judgement. Cooper [Bef. 1 
weights through subjective 
regression. Augood [Ref. 
0/ 5f Z, and 10 to hi 

descriptors and subjective 
weights to each element, 
score as 

PVj = (100 Sum/i [ (lii) {Si 

where the previous nctatio 
form bounds the score be 
success. Scoring models hav 
in a number of organization 
[Bef. 7], with various 
utilized. 

c. Index Models 

These models 

project success and costs, 
model is; 



Sij are normally assigned by E5D 
roups based on personal or group 
8] empirically derives criteria 
survey data and a multivariable 
8] suggested applying the weights 
s five increasing benefit element 
ly assigning relative importance 
He then represented a project 

j) ])/ (10 Sum/i [ Wi]) (3.2) 

n of equation 3.1 is used. This 
tween 0 as failure to 100 as 
€ been used for project selection 
s, according to Winkofsky# et al. 
criteria and weighting schemes 



directly involve probability of 
The general form of an index 



I = (P) (E) / C (3.3) 

where , 

I = project index value, 

P = project probability of success, 

B = project benefit, and 
C = cost. 

The differences in the index models proposed in the litera- 
ture are determined by what is included in the three model 
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paraiE€ters. The probability of success nay include 
subfactcr probabilities such as success in research, tech- 
nology, production, and market areas. Benefits include 
items such as savings, profits, or cash flow. Costs range 
from BSD investment tc total lifecycle expenditures. These 
indices may be used alone or combined into other indices to 
provide the final prcject measurement such as the Anscff 
index, suggested by fiinkofsky, et al. [Eef. 7]) as typical 
cf BSD project selection indices. Ansoff's index provides a 
numerical value called the Figure of .^erit (?M) such that 

FM = rdp ( T + B )i / (Total investment) (3.4) 

where, 

r = probability cf research success, 
d = probability cf development success, 
p = probability of market success, 

T = index of technical merit, 

B = index of business merit, and 
E = present value cf earnings from project. 

like the scoring models, the index models combine a number 
of prcject attributes into a single value which may then be 
used to rank projects and assist in determining prcject 
selection. More cost analysis is required for typical index 
models than is generally necessary for scoring models. 

2 - Por tfo lio Sele ctio n Mo dels 

Portfolio selection models are normally more compli- 
cated than benefit measurement models; however, they also 
provide additional flexibility and realism to the BSD deci- 
sion maker, according to Souder [Hef. 17]. Typically these 
models utilize a mathematical programming technique to 
assist in project selection by providing the resource 
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allocation that will iraximi ze the benefit contribution for a 
set of possible projects. The models are classified within 
the portfolio selection category by the mathematical 
programming form used. Linear, non-linear, integer, and 
dynamic programming model forms have been proposed in 
literature. Winkofshy, et al. [Ref. 7] observed in their 
review that these portfolio selection models assume that an 
explicit benefit or contribution function can be determined. 
They reviewed other authors, however, that utilize an inter- 
active programming model form which assumes an implicit 
benefit function exists but is never formally defined. 

a. Linear Programming 

The general form of the linear programming model 
is 

maximize cx (3.5j 



subject to Ax < b 

0 < X < 1, 

where , 

X = n-ccmponent project resource allocation vector, 
c = n-component project benefit vector, 

A = mxn-componen t project resource utilization matrix, and 
b = m-component available resource allocation vector. 

linear programming ccirputations provide an optimal resource 
allocation for all projects provided that the values repre- 
sented by c, b, and A can be explicitly obtained and the 
assumed linear relationships between resources and benefits 
portray an accurate model of the using organization's situ- 
ation. The project resource allocation vector x, provided 
by the computations, takes values between and including zero 
and one. Zero represents no resource allocated on that 
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specific project ard a one represents full allccaticn. 
Values tetween zero and one provide the proportion of full 
resource allocation necessary for the optimal scluticn. 
Asher £Eef. 19] proposed a linear programming model to allo- 
cate manpower teams involved in the testing operation of new 
chemioal compounds urder development by a pharmaceutical 
company. The project resource allocation vector represented 
the number of tests of a specific compound each team would 
conduct and was bounded by the total number or each chemical 
compound available. 2 ihe expected return of the compound to 
the firm was used as the the project benefit vector and was 
calculated as the product of the probability of success and 
the estimated net market value of success for each compound. 
The project resource utilization matrix was equal to the 
number of hours necessary for each team to test a specific 
compcurd and was based on the team’s experience and skill 
levels. The resource vector was the manhours available for 
each team. The computation results provided which manpower 
team should test which compound and the quantity of that 
compound they should test in order to maximize the expected 
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equations 3.5 and provide similar resource allocation infor- 
mation. A non-linear benefit function has the intuitive 
appeal that as more units of resource are provided to a 
project less benefit is returned for each additional input 



2The bound of one was previously noted for the typical 
linear program model. Asher's computations found the actual 
allocation value enstead of a proportion. 
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unit. Chidambaram [Ref. 20] considered three separate 
concave nondecreasing functions; g[uadratic, exponential, and 
logarithnic ; which all approximated his U.S, Army data of 
estimated future military benefits for a set of projects. 
Souder [Eef. 16] proposed and analyzed four portfolio selec- 
tion models based on similar models in the literature using 
thirty actual R5D projects. For his model Souder used two 
types cf ncn-linear benefit functions which are represented 
by Figures 3.1 and 3.2. The exponential type. Figure 3.1, is 
also similar to Chidambaram's benefit functions. Scuder 
suggests using this type of function when incremental 
resource expenditures in the lower x domain are expected to 
greatly increase the return, and the benefit diminishes at 
some point as resource levels are increased. The S-shaped 
benefit function. Figure 3.2, was suggested for project 
cases where success was related to a breakthrough in tech- 
nology. In this case the return from the resource input was 
initially expected to be lower until the breakthrough. Then, 
small additional resource expenditures returned larger 
benefits until a diminishing return point is reached. 




Figure 3.1 Exponential Type of Benefit Function. 
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Figure 3.2 S-Shaped Type of Benefit Function. 



c. Goal Programming 

Goal programming is a mathematical programming 
method vhich incorporates several objective functions as 
organizational goals and attempts to solve them simultane- 
ously. The objective or goal functions become the 
constraints of a linear or non-linear programming problem. 
The problem's overall objective function is a composite of 
the goal functions which are weighted in their relative 
accomplishment importance by the program user. The computa- 
tion minimizes the prioritized deviations from the goal 
constraints as described by the composite objective function 
and defines a ranee of feasible solutions between the 
competing goals. Goal programming in RSD models is normally 
used with other model forms such as in Taylor, et al. 
[Ref. 21] and Rinkofsky, et al. [Ref. 12]. 

d. Integer Programming Models 

Integer programming models are formulated in a 
similar manner to the linear and non-linear models, however 
the project resource allocation vector is constrained to be 
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integers only. Taylor, et al. [Ref. 21] utilized an 
integer program model incorporating goal programming methods 
to allocate thirty researchers among seven P.SD projects. 
Non-linear constraints were used to relate the number of 
researchers assigned to a project to probability of success, 
monetary return, ard project time completion. Linear 
constraints were used to describe budget limitations and 
computer capacity utilization. A linear composite objective 
function was used that described prioritized deviations from 
each goal constraint and the integer program computation 
minimized these deviations to provide the optimal allocation 
solution. Other authors have proposed simpler 0-1 integer 
program models which provide a go or no-go project decision 
depending on the resource allocation vector values of cne or 
zero respectively. 

e. Dynamic Programming Models 

The first three portfolio selection model forms 
described are closely related and differ only in the benefit 
and constraint functional relationships and the domain of 
the resource allocation vector. ■ A dynamic programming 
approach to BSD project selection provides a different opti- 
mizing technigue with a seguential decision process. Hess 
[Ref. 22] presents such an approach as more typical of the 
E8D budgeting problem than a current-cycle-only optimizing 
solution. Hess suggests that most R&D resource allocation 
models do net adequately consider the periodic re-evaluation 
of RSE projects which stem from the increase in information 
obtained during the ESD process as well as the cyclic budg- 
eting evolution. Hess proposed using the mathematical 
recursive technique of dynamic programming to develop an 
optimal resource allocation policy, ?o, for the series of 
subsequent project resource decisions stages. The general 
recursive equations for Hess’ model are 
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X = initial quantity of resource, 

P = policy or series of resource decisions used, 

xn-1(x,P) = resource function of x and ? at stage n-1, and 
Rn(x,P) = benefit returned in stage n using x and P. 

Assuining En(x,p) is known through some benefit measurement 
method, the optimal policy, Po, is calculated by first 
solving equation 3.6 for the last decision, stage 1. Next, 
equation 3.7 is solved for n=2 with the substitution of 
equation 3.6 The value of n is increased and equation 3.7 is 
solved again with the substitution of the n-1 results. Each 
incremental increase in n represents the next prior decision 
stage in the sequential R&D resource allocation process. The 
dynamic programming model is dependent on knowing the 
benefit values of Ri(x,P) (i = n , n- 1 , . . . , 2, 1 ) . This requires 

project information representing the entire project R5D 
phase and the number of cycles remaining until project RSD 
completion. This information is normally based on subjeotive 
evaluations and estimates which are used to derive the 
resource to benefit relationships. Gear, et al. £Ref. 9] 
analyzed several dynamic programming models similar to and 
including Hess’ model. They observed that this form of the 
model has the advantage of catering to the multistage 
learning and decision nature of RSD projects. Although this 
aspect is referenced throughout the literature, the model 
forms presented above do not consider it as well as the 
dynamic programming form. 
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f. Interactive Programming Models 



Winkofsky, et al. £Ref. 7] reviewed several 
authors that proposed interactive programming models as 
solution techniques for multiple criteria problems. These 
models can provide helpful tods in the RSD portfolio selec- 
tion process. Zoints and Wallenius [Ref. 23] proposed a 
lan-mach’ine interactive programming method which allows a 
decision maker to optimize an implicit benefit function 
involving multiple objective or goal functions. The benefit 
function is a composite objective function and is unknown to 
the decision maker; however, it is assumed to be a linear 
function, or in a more general form a concave function, of 
several known goal functions. The implicit composite objec- 
tive function is maximized against a set of convex 
constraints through the decision maker's answers to yes-or- 
no questions involving trade offs between possible goal 
function solutions. Zoints and Wallenius' method initially 
optimizes a computer generated composite objective function 
including the goal function constraints and an arbitrary set 
of function multipliers. The technique produces a 
Paret c-opti nal solution to the problem. Next a subset of 
nonbasic variables is generated that, if introduced into the 
basis, would continue to yield a Pareto-optimal solution. 
For these selected variables a series of trade offs are 
defined which increase or decrease each of the goal func- 
tions and the trade offs are presented to the decision 
maker. The decision maker's responses to the trade offs are 
used to generate a new set of goal function multipliers. The 
new multipliers develop a new composite objective function 
closer to the implicit function. The authors state that 
successive iterations of this process assures convergence to 
the optimal solution. Their premise is that all 
Pareto-optimal solutions form a subset of all extreme point 
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solutions and with each 
is eliminated from the 
Since there is a finite 
be a finite number of 
optima] solution. 



iteration a Pareto-optimal solution 
set of possible optimal solutions, 
number of extreme points there will 
iterations required to obtain the 
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12] proposed a decision process 
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1 constraints. The lowest level 
goal program and provides the 
middle level. The middle level 
ncorporating all subordinate unit 
onal constraints from the higher 
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B. AKAiyTIC HIERARCBl PROCESS 

Saaty [Ref. 24] proposes the 
Process" (AHP) as a decision tool for 
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acd making trade-offs. Saaty provides many possible applica- 
tions for this process of which RSD resource allocaticn is 
one. Ihrcugh structuring a hierarchy of criteria stake- 
holders and outcomes and by developing judgemental based 
priorities the process contributes in complex problem 
solving and predictions of likly outcomes. AKP uses three 
principals of analytic thinking; structuring hierarchies, 
setting priorities, and logical consistency. AHP enables 
consideration of a problem as a whole and the study of the 
component interactions within a hierarchical problem 
description. Saaty describes an eight step process which 
encompasses a graphical breakdown of the problem into a 
decision tree hierarchy, involves pairwise criteria compari- 
sons, establishes decision priorities, and evaluates the 
consistency of the comparisons. Figure 3.3 shows his example 
hierarchy structure for choosing E&D projects to ensure 
adequate future power and electricity. The figure lists six 
levels of hierarchy, one focus or central objective and the 
remaining levels each with several elements. The process 
involves a pairwise comparison starting from the top. A 
higher level is used as the comparison property for the next 
lower level. The pairwise comparisons of the elements under 
a property become the basis for the computation of relative 
element priorities. Finally, the consistency of the compari- 
sons is investigated. 

E. MCEEl ACCEPTABILITY AND ANALYSIS 

The acceptability of RSD project selection models by 
industry and government has long been a problem. This is 
evident from statements in the majority of articles 
reviewed. Baker [Ref. 6] lists seven limitations of proposed 
RSD models based on his and others’ research. Baker’s 
summary of RSD model limitations is as follows: 
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Figure 3.3 RSD Hierarchy Exaaple. 
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1) Iradeguate treatment cf project and parameter 
interrelations with respect to both Benefit contribution 
and to resource utilization. 

2) Inadequate treatment of uncertainty as it impacts in 
benefit measurement and parameter estimation. 

3) Inadequate treatment of multiple, interrelated deci- 
sion criteria which have no underlying measure. 

4) Inadequate treatment of the time variant property of 
the parameters and criteria and the associated problem 
of continuity in tie research program and staff. 

5) A restricted view of the problem which (a) portrays a 
once-a-year investment decision rather than an intermit- 
tent stream of investment alternatives, (b) does not 
include such attributes as timing of the decision, 
generation of additional alternatives, and recycling, 
(c) dees not recognize the diversity of projects alcng 
the spectrum from Basic research to engineering, and (d) 
views the problem as a decision event rather than a 
hierarchical, diffuse decision process. 

6) Nc explicit recognition and incorporation of the 
importance of individual RSD personnel. 

7) The inability to establish and maintain balance in 
the RSE program; e.g., balance between basic and applied 
research, between offensive and defensive research, 
between breakthrough and improvement orientations, 
between in-house and contracted projects, between 
product and process oriented projects and between high 
risk/high payoff and low risk/moderate payoff projects. 



Baker continues with the conclusion that this list cf limi- 
tations makes it clear why few RSD models have been imple- 
mented for use by RSD managers and, in the cases where 
models are utilized, the majority are the simpler scoring or 
index form. 

Souder [Ref. 17] developed a scoring model to assess the 
suitability of R8D models. His criteria used for the model, 
based on his interviews with management scientists and RSD 
administrators, are presented in Table II. Souder then used 
his scoring model to rate 26 models for use in RSD project 



selection decision support, 
were again found to be in 
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TABLE II 

Five Model Criteria and Their Characteristics 



Criteria 

HEALISM 



FLEXIBILITY 



CAPABILITY 



OSE 



CCST 



Cha rac t er i s ties 

model includes: 

Multiple Objectives 
Multiple Constraints 
Market Risk Parameter 
Technical Risk Parameter 
Manpower Limits Parameter 
Facility Limits Parameter 
Budget Limits Parameter 
Premises Uncertainty Parameter 

model applicable to: 

Applied Projects 

Basic Projects 

Priority Decisions 

Termination Decisions 

Initiation Decisions 

Budget Allocation Application 

Project Funding Application 

models performs: 

Multiple Time Period Analyses 
Optimization Analyses 
Simulation Analyses 
Scheduling Analyses 

model is characterized by: 

Familiar Variables 
Discrete Variables 
Computer Not Needed 
Special Persons Not Needed 
Special Interpretations Not Needed 

model has: 

Low Set-up Costs 

Low Personnel Costs 

Low Computer Time 

Low Data Collection Costs 



F. SUMMARY 

This chapter has reviewed 
model forms found through out 
presented were typical of the 



several RSD project selection 
the literature. The models 
literature and were the more 
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general lorus which cculd have application in the following 
analysis. The comincnality ohserved between model forms is 
the use of subjective project benefit relationships in seme 
manner. Decision processes and the Analytic Hierarchy 
Process were briefly presented as methods for approaching 
the ESD project selection problem. Table III summarizes the 
models presented in this chapter. Other proposed project 
selection models are listed in the bibliography. The next 
chapter describes a R&D program prioritization process 
proposed within the Marine CorpS/ and classifies the process 
model form according to the literature. 
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TABLE III 


Summary of Eeviewed Models and References 


Authci/Beference 


Form Descriptor Used 


Aster (1S62) 
[Ref. T9: 


linear Programming 


Augood M973) 
iRef. 8] 


Checklists 

Simple 

Quantified 

Profile 

Indices 


Baker MS74) 
[Ref. 6] 


Benefit Measurement 
Comparative 
Scoring 

Benefit Contribution 
Project Selection/Allocation 
Ranking 

Linear Programming 
Non-linear Programming 
Dynamic Programming 
Integer Programming 


Chid aitaram (1970) 
[R€f. 20] 


Ncn-linear 


Cccper (1981) 
[Ref. 18] 


Empirical (scoring) 


Hess (1S62) 
[ R€f . 22 ] 


Dynamic Programming 


Scuder (1973) 
[Ref. 16] 


Profitability Index 
linear 
Non-linear 
Zero-One 


Taylor et al. (1982) 
[Ref. 21] 


Integer Ncn-linear 
Goal Programming 


Winkofsky et al. (1980) 
[Ref. 7] 


Value Measurement 
Checklist 
Scoring 

Economic Index 
Portfolio Selection 
Linear Programming 
Integer Programming 
Nonlinear Programming 
Interactive Programming 


Winkofsky et al. (1981) 
[Ref. 12j 


0-1 Goal Programming 


Zcints et al. (1 976) 
[Ref. 23] 


Interactive Programming 
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IV. MCIEC PRIOEIIIZATION PRO^SS 



a. INIECDDCTIOH 

The KCDEC Prioritization Process consists of four tine- 
dependent phases synchronized with the Marine Corps partici- 
pation in the DOD Planning Programming and Budgeting System 
and is organized fcr implementation at the Marine Corps 
Development Center (EevCtr) , MCDEC. Figure 4.1 depicts this 
synchronization and the process timetable for a single 
cycle. The four phases include determining deficiencies 
and requirements, prioritizing deficiencies and require- 
ments, prioritizing research and development programs, and 
completion of funding profiles. This process, as described 
by Major J. L. Creed, USMC, in "A Guide for the Performance 
of the Development cf the MCDEC BSD Program Prioritization 
Process, Methodology Manual" [Ref. 25], is carried out under 
the responsibility of the Office of the Deputy Chief of 
Staff (DC/S), Developmental Coordination (DC), DevCtr, 
MCDEC, and ty two primary groups, the Prioritization Working 
Group (PEWG) and the Chief of Staff's (C/S*s), DevCtr 
Prioritization Committee (Pri Com) . Each Division and 

Directorate of the EevCtr provides a representative for 
membership in the PEWG. The Division and Directorate Heads 
compose the C/S ' s DevCtr Prioritization Committee. Appendix 
A details the phases and tasks of the process and lists the 
EevCtr Divisions, Directorates, and Developmental 

Coordination Branches responsible for completing each step. 
This chapter describes the process concepts and concentrates 
on the participants as shown in Figure 4.2 . 
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Figure 4. 1 HCEEC Process and EDTSE POM Cycle. 
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Figure 4.2 MCDEC Process Participants. 



E. PEOCESS PHASES 



1 . Phase I 



The determination of deficiencies and rs guirements, 
phase I, involves three general tasks and serves to provide 
the PPMG with an infcrmation base for the following subjec- 
tive evaluations. First, the Marine Corps Midrange 
Objectives Plan (MMROP) and the Marine Corps Long Range Plan 
(MLEP) are updated utilizing Defense Guidance (DG) , the 
FYDP, previous POM recommendations, and other appropriate 
research and developaent information. 
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Second, Fleet Marine Force (FMF) input is solicited 
hy survey analysis through cuestionnaires, opinionnaires, 
and personal interviews. This task, provides the ever.tual 
systeit users with an opportunity to input observations of 
mission deficiencies, ideas for new programs, and their 
rankings of ongoing programs. 

Finally, MAAs are conducted which provide the frame- 
work necessary for phase II and serve to formalize all defi- 
ciencies that diminish the capability to perform the mission 
area. The MAAs are classified according to the Mission Area 
Elements (MAEs) listed in Table I, and performed by project 
officers under the direction of the Planning and Evaluation 
Eranch, DevCtr. The assigned project officer conducts his 
analysis through a series of mission categorizations which 
are based on the previously updated plans and other estab- 
lished Marine Corps Doctrine. Three basic categorizations 
occur during the MAA. First, the mission area element is 
defined into its general and specific operational mission, 
then the resources necessary to accomplish the operational 
missions are identified, and finally each resource is 
analyzed against each operational mission to identify if any 
deficiencies exist. The collection of these identified 
mission deficiencies present the recommended corrective 
actions to obtain full Marine Corps capability in a mission 
area . 

2. Phase II 



Phase II merges the identified mission element defi- 
ciencies into a single prioritized list. To accomplish this 
ranking, the PHWG first receives briefings concerning the 
MAAs and survey analyses from phase I. Next, the PAWG 
constructs a decision tree based on tne presented informa- 
tion to order deficiencies within mission area elements. 
The tree development is represented by Figures 4.3 through 



44 



4.6. This development parallels the mission element categcr- 
izaticE process conducted during the MAAs and applies multi- 
attrihcte utility analysis to the categories as explained by 
Creed [Eef. 25]. The subsequent evaluations of the elements 
and Eissicns, represented by the decision tree branchs, also 
rely heavily on the MAAs as presented to the PRHG by the 
project officers, and by the Planning and Evaluation Branch. 
Additional technical assistance for the decision tree 
construction and analysis is provided by the Analysis 
Support Branch. 

The decision tree is based in the USMC mission areas 
and elements listed in Table I and is represented in Figure 
4.3. The tree branches through the doctrine based general 
and specific missions for each element (see Figure 4.4), and 
then these specific missions are shown supported by the 
necessary resources, as presented in Figure 4.5. 



USMC MISSION 
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Kote: See Table 3 



Figure 4.3 Eecision Tree, Established Base. 
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Decisicn Tree, Resources. 



The resources shown in Figure 4.5 are further 
defined into principal and ancillary categories. Principal 
resoores ccrrespcnd tc equipment or weapon systems that are 
used to accomplish the specific mission directly, and 
ancillary resources are necessary but only indirectly 
support the mission, a common list of resources necessary to 
accomplish the specific operational mission is initially 
applied to all specific operational missions with seme 
resources subsequently being determined not applicable for 
all missions. As an example. Creed [Ref. 25] suggests a 
possible resource list for the close combat mission area 
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Figure 4.6 Decision Tree# Deficiencies. 

element to te direct fire, assault, and anti-armor weapons 
for the principal resource category and communication equip- 
ment and target acquisition devices for the ancillary 
resources. The resource deficiencies and, thus, the mission 
deficiencies shown in Figure 4.6 are identified by evalu- 
ating each resource as it supports the a^ccomplish ment of the 
specific operational mission. 

The PRWG assigns subjective importance values^ (0.01 
to 1.00) , to each branch of the decision tree from the 
mission area through the deficiency level. These values are 
based on presented information and finalized through collec- 
tive group judgement. The Mission Area Values (MAV) repre- 
sent the relative importance of each mission area within the 
USMC mission and the Mission Area Element Values (MAEV) 
represent the relative importance of each mission area 
element within the mission area. Similarly, General 
Operational Mission Values (GOHV) , Specific Operational 
Mission Values (SOMV), and Deficiency Values (DV) represent 
the relative importance of each general operational mission 



3Thesf values represent linear relative importan<;e, (ie, 
a .74 deficiency is twice as critical as a .37 deficiency). 
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vithir the mission area element, each specific operational 
mission vithin the general operational mission, and each 
deficiency within the specific operational mission respec- 
tively. The initial value sets, IIAZV, GOMV, SOMV, and CV, 
are normalized to a common tase for each set which allow 
comparisons across all elements, missions, and deficiencies 
in their respective decision tree level. The Deficiency 
Eelative Importance Weight (DEIW) for each specific mission 
deficiency is defined as 



DEIW = MAV • MAEV • GOMV • SOMV • DV (h.1) 

where , 

MAV = Mission Area Value, 

MAEV = Mission Area Element Value, 

GCKV = General Operational Mission Values, 

SOMV = Specific Operational Mission Values, and 
DV = Deficiency Value. 

The EEIWs, in turn, determine a mission deficiency rank 
ordering . 

3. iijase III 

Ehase III prioritizes ongoing EDTSS acquisition 
programs through presentation of the prioritization group 
and committee with available information, assigning program 
subjective values, deriving program effectiveness, combining 
previous program prioritizations, and presenting the results 
for Director, DevCtr approval. First, the Prioritization 
Workirg Group receives briefs by the Developmental Project 
Officers (DPOs) to assist in determining the worth and 
utility of the program. Next, survey analysis briefings 
provide the FMF evaluations of current EDT&E programs. This 
information presented in the PEWG briefs is also provided 
separately to the DC/S DC Prioritization Committee. 
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Ihe next task of phase III is the subjective priori- 
tization cf EDT5E programs. During this task, a sample of 
experts are solicited for their desired RDT&E program rank 
orderings and relative worth values (0.01 to 1.00). This 
opinion sample consists cf the DC/S DC Prioritization 
Committee membership, minimally, and may include other qual- 
ified individuals. The prioritization is performed using an 
analytic Delphi process where the independent rank ordering 
and worth values are combined and returned to the sample 
membership for study and re submission . After at least three 
replications with relatively common consecutive listings, 
the prioritization concludes with a program ranking and 
program Subjective Evaluation Values (SEVs) . 

Following the subjective evaluation the EDTSS 
programs are prioritized based on their mission deficiency 
effectiveness. The PEWG identifies all deficiencies within 
each mission element that are directly affected by a EDT&Z 
program, and it evaluates the proportion at which the 
program accomplishes the specific deficiency. This propor- 
tion is defined as the Program’s Proportion of Deficiency 
Accomplishment (PPDA). The evaluation process relies 
heavily cn the input from the DPOs concerning each program's 
proposed operational characteristics. The product of the 
specific DEIW, assigned in phase II, and the specific PPDA 
are summed, over all deficiencies, to obtain a program 
Deficiency-Derived Efficiency Value (DDEV) as 

DDEV = Sum/d [ (DE IK) • (PP DA) ] (4.2) 

where , 

SuE/d = Summation ever all deficiencies, 

DEIS = Deficiency Relative Importance Weight and 

PPDA = Program's Proportion of Deficiency Accomplishment. 

The prcgiams are then ranked according to the DDEV values. 
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lh€ PRWG obtains a final EDTSZ program ranking by 
combining the subjective and deficiency-derived prioritiza- 
tions. A Subjective Evaluation List Weight (SELW) and a 
Deficiency-Derived List Peight (DDLW) is assigned to the 
appropriate program list combining them in a linear rela- 
tionship based on PBPG consensus of each list's validity. 
Ihe sum, by program, of the weighted prioritized lists 
provides the Final REISE Program Value (FPV) as 



FPV = (SELP)*(SEV) + (DDLW) • (DDEV) 



( 4 . 3 ) 



where , 



Ihe 

the 



SEIW = Subjective Evalua 
SEV = Subjective Evalua 
DDIW = Defi ciency-E erive 
DDEV = Deficiency-Eerive 

final prioritized rank 
FPV and this completes 



4 . Phase IV 
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ing is accomplished according to 
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and proposing funding profiles 
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er file for the next pricritiza- 
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E PCM development process. Creed 
development in this phase which 
urrently, these funding profiles 
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chapter concludes with classifying the model form used in 
the MCEEC process as a benefit measurement scoring model, 
according tc similarities with proposed models in the liter- 
ature. The literature suggests other model form applications, 
model limitations, and model evaluation criteria which will 
assist further investigation cf possible improved alterna- 
tives or enhancements to the MCDZC prioritization process. 



TABLE IV 
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V. A MCDBC PORTFOLIO SELECTION MODEL 
A. INTECDDCTION 

The MCDZC process preserted in the previous chapter was 
identified as a benefit measurement scoring model with 
similar characteristics as models currently used in 
industry. The literature also proposes portfolio selection 
models which use benefit measurement relationships to opti- 
mize resource allocation through various mathematical 
programming techniques. The MCDEC process, as proposed, 
does not attempt to optimize program resource allocation 
and, thus, the obvious first process improvement to be 
investigated is the extension of the current model into a 
portfolio selection model. The simplest and most comprehen- 
dable way This can he accomplished is by formulating the 
process as a linear programming model. This mcdel form 
approxiirat es resource and benefit relationships with 
straightforward linear functions that are easly understood. 
The linear programming model is the basic st;:ucture for 
other more complicated portfolio selection models and will 
provide a base for observing advantages from a process model 
change. The primary purpose of the following linear program 
development is to illustrate the feasibility, flexibility, 
and required assumptions of a portfolio selection mcdel as 
applied to the MCDEC prioritization process and does not 
include all constraints and relationships required for a 
complete RBT&E funding allocation. 

E. FCEMDLATION 

The initial step of any linear programming formulation 
can be viewed as a thought exercise. In this exercise the 
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(PPDA), aad the Subjective Evaluation Value (SEV) . These 
three values are ccmbined in equations 4.2 and 4.3 to 
provide the Final RDISE Program Value (FPV) . Resource data 
is ret defined in the current process but is considered 
during Project Develcpment Officer briefs to the prioritiza- 
tion group and cemmittee. The resource data is also used in 
phase IV to provide suggested RDT5S,Navy POM input. Previous 
POMs that are independent cf the process are used as proxy 
resource estimators for illustration in the following 
formulations. 

The decision variables for the formulations are implied 
from the desired output. The following series of linear 
programs will be formulated using the decision variable Xi 
which represents the proportion of the ith program that is 
funded or Xik which represents the proportion of the ith 
program that is funded in the kth time period. The values 
Xi and Xik are defined between and including zero and one. 
A variable, Yi, which represents the proportional deviation 
from a fixed jth deficiency accomplishment goal is used in 
later linear programming formulations and is also defined 
between and including zero and one. 

As in the literature, the objective functions for the 
following formulations are provided from the benefit meas- 
urement model. For the initial linear program formulation 
the Final Program Value (FPV) vector from the MCDEC process 
is used. This vector represents a combination of deficiency 
derived and subjectively assigned mission deficiency accom- 
plishment program values. The vector FPV contains a value 
FPVi for each program, where i is an integer from 1 to p. 
Formulation 5.1 provides an initial unconstrained linear 
programming model that is equivalent to the MCDEC benefit 
measuremient model and can he solved by observation. This 
linear program maximizes the total relative program impor- 
tance as individually defined by each FPVi, reduced by the 
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available prograic funding proportion Xi, and summed ever all 
programs. The optimal solution for this for mulaticr. is 
obtained when Xi is set at one for all i. The initial fermu- 
laticn is 

luaximiize Sum/i [ (FPVi) (XiJ ] (5.1) 

subject to: 0 < Xi < 1 

i = 1 .. . p. 

Formulation 5.1 may be trivial but it provides a linear 
program structure eguivalent tc the MCDEC process which can 
be constrained by budget or deficiency relationships. 
Formulation 5.2 provides a constrained version of formula- 
tion 5. 1 as 



maximize Sum/i [ (FPVi) (Xi) ] 



(5.2) 



subject to 



Sum/i [ (Aki) (Xi) ] < Bk 

0 < Xi < 1 

1 = 1. . . p 
k = 1...t 



where , 
FPVi 
Xi 
Bk 
Aki 

P 

t 



ith final program value, 
proportion of ith program 
available budget resource 
full funding level of ith 
number of programs, and 
number of time periods. 



funded, 

in time period k, 
program in time period k. 



If Bk is greater than or egual to Aki, summed over all 
programs for each time period, the optimal solution to 
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formulaticn 5.2 is also at Xi ei^ual one for all i. This 
formulation, however, will provide a tool to observe the 
optimal allocation change when the budget level Bk is 
reduced, thus constraining the objective function. The new 
optimal program priority listing will not necessarily be the 
same as the ordering provided by FPV. After the linear 
program computations, the vector (FPV) (X) represents the 
proportional reduction in the programs' Final Program Value 
and provides the revised priority listing under budget 
constraints . 

Formulation 5.2 is illustrated by an example. Table V 
lists the benefit measurement output values from four PDTSE 
programs considered during a partial HCDEC process test 
conducted by the Development Center. The respective budget 
allocation data from the RDTSE POM is also listed. A 
constant is imbedded into the linear program formulation 
which defines possible budget reductions and thus a tighting 
of the constraints. The constant is added by replacing the 
budget ccnstraints of formulation 5.2 with 

Sum/i [ (Aki) (Xi) ] + BR(Bk) < Bk (5.3) 

wher € , 

BR = budget reduction proportion. 

Eight progressivly reduced budget conditions are used in 
this example. 

Table VI lists the resulting optimal solutions and rank- 
ings for each of the eight budget conditions. The proportion 
of full funding for each project changes but not as the 
MCDEC prioritizing vector FPV implies. Although the FPV of 
program C0020 is larger than the FPV for both programs C0021 
and CC082, the latter two programs are fully funded at each 
budget reduction. Tbe values of FPV alone would not suggest 
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TABLE V 

HCDZC Portfolio Selection Model Example ®ata 



Program FPV Budget Data 







yr 1 


yr 2 


yr 3 


yr 4 


yr 5 


Cl 120 


7.32464 


4747 


20874 


9176 


8987 


8988 


C0020 


.6.9616 


21100 


22200 


30100 


33700 


7C500 


C002 1 


4.79723 


4172 


4297 


4369 


4268 


3927 


C0C82 


4.2 19 85 


361 


370 


380 


442 


443 


Budget 


Sum =Bk= 


30380 


47741 


44025 


47397 


83858 



these funding reductions. For this example/ observation of 
the differences in the programs' budget profiles as listed 
in Talle V would implj that the optimal funding in a budget 
constrained problem would be obtained by reducing the most 
expensive program first. However, in a larger example the 
optima] solution may not be as evident. 

Formulation 5.2 can be expanded to include a time index 
for the funding proportion variable X. This expansion is 
formulated as 

maximize Sum/k [Sum/i (FPVi) (Xik) ] (5.4) 

subject to 

Sum/i [ (Aki) (Xik) ] + BE (Bk) < Bk 
0 < Xik < 1 
i = 1 . . . p 
k = 1. . . t 

where , 

BE = budget reduction proportion, and 

Xik = proportion of ith program funded in the kth period. 
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Ihe data values for PIVi, Aik, and Bk remain the same as in 
formulation 5.2. Formulation 5.4 provides greater flexi- 
bility in funding level possibilities and provides a 
sequence of program rankings for each subsequent year. Table 
VII provides a summary of the funding proportion solutions 
and the ranking according to the vector (FPV) (X) for each 
year and budget condition. The most expensive program C0020 
is again the only program that requires less than full 
funding except in the most extreme condition where program 
C1172 is also less. Observing the proportional funding 
profile of program C0020 implies that the first time period 
is the most restrictive in each budget condition. 

Considering funding levels in each time period allows a 
greater utilization of resources available in the less 
restrictive periods. 



TABLE VI 





For 


mulation 5. 2 Solution Results 
Proportion of Full Budqet 






Vector X 


1 .0 


.99 


.95 


. 90 


.85 


: 80 


.75 


.50 


C1120 XI 


1 . 0 


1. 0 


1.0 


1.0 


1.0 


1.0 


1.0 


.92 


C0020 X2 


1 . 0 


.979 


.892 


. 784 


.677 


.57 


. 462 


0. 0 


C0021 X3 


1.0 


1 . 0 


1.0 


1 . 0 


1.0 


1.0 


1.0 


1.0 


C0C82 X4 


1.0 


1 . 0 


1.0 


1.0 


1 . 0 


1.0 


1.0 


1. 0 


BE 

Ranking 


. 00 


. 0 1 


.05 


. 10 


. 15 


. 20 


,25 


.50 


C1120 


1 


1 


1 


1 


1 


1 


1 


1 


C0C20 


2 


2 


2 


2 


3 


4 


4 


4 


C0021 


3 


3 


3 


3 


2 


2 


2 


2 


C0082 

Notes 


4 


4 


4 


4 


4 


3 


3 


3 



1. X = Proportion of program funded and is specific 
for each budget reduction case. 

2. BR = Proportion of budget reduction. 

3. Ranking is according to the vector (FPV) (X) . 
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TABLE VII 

Formulation 5- 4 Solution Results 



Prcporticn cf Full Budget 
1.0 -99 .§5 .90 -85 . 80 -75 . 50 

Xik Matrix 

C1120 



yr 1-5 


1 .0 


1 .0 


1.0 


1. 0 


1. 0 


1.0 1.0 


.921 


C0020 


yr 1 


1 .0 


.97 9 


.892 


.784 


.677 


. 57 .46 2 


0.0 


2 


1 .0 


. 985 


.927 


.8 54 


.780 


. 707 .634 


.269 


3 


1.0 


. 986 


.93 


.859 


.789 


.719 .648 


.298 


4 


1 .0 


. 98 8 


.94 


. 881 


.821 


.762 .702 


. 405 


C 


1.0 


- 986 


.928 


- 856 


.784 


-712 .640 


.280 


C0021 


yr 1-5 


1.0 


1 .0 


1.0 


1. 0 


1. 0 


1.0 1.0 


1.0 


C0C82 


yr 1-5 


1.0 


1 .0 


1.0 


1.0 


1.0 


1.0 1.0 


1-0 


BR 


.00 


. 01 


-05 


. 10 


.15 


.20 .25 


.50 


Ranking 


C 1 120 


1 


1 


1 


1 


1 


1 1 


1 


C0020 


2 


2 


2 


2 


2 


2 3 


4 


C0021 




3 


3 


3 


3 


3 2 


2 


C0C82 


4 


4 


4 


4 


4 


4 4 


3 



Notes 

1. Cl 120 is funded at .92 for y'ear one only and at 
1-0 for the years two through five. 

2. Xik = Proportion of program funded and is specific 
for each year and tudget reduction case. 

3. EE = Proportion of hudget reduction. 

4. Ranking is according to the vector 
(FEV) (Sum/yx (X)) . 



The Final Program Value (FPV) provided by the MCCEC 
process as a benefit measurement is postulated to represent 
the relative program importance in accomplishing mission 
deficiencies. The primary purpose of defense systems acqui- 
sition and Marine Ccrps RDTSE as presented previously in 
Chapter II, is the accomplishment or alleviation of 
mission def iciencies . Concentration on a portfolio selec- 

tion model which considers only the FPV and budget 
constraints may net produce solutions representing the 
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greatest accomplishmert of individual mission deficiencies. 
Ihe next two proposed formulations consider missicn defi- 
ciency accomplishment in terms of other dCDEC output values 
under separate constraining conditions. A third additional 
formulation concentrates directly on the number of deficien- 
cies alleviated. 

Formulation 5.5 optimizes at a level in the «CEEC 
benefit measurement model where each deficiency is consid- 
ered individually. The FP7 is constructed from the previ- 
ously presented values of SEV and DDEV. The DD EV directly 
considers each deficiency where the SEV is the subjective 
evaluation of the RDTSZ programs only. Equation h.2 defines 
the vector DDEV as the product of the Deficiency F.elative 
Importance Weight (EFIW) and the Program Proportion of 
Deficiency Accomplishment (PPDA) summed over all deficien- 
cies. Letting the vector PPDA be linearly reduced by the 
funding proportion of the respective program provides a 
means to optimize the missicn deficiency accomplishment 
directly. The linear program to maximize the EDTSE program's 
DDEV is formulated as 



maximize Sum/i [ (IDEVi) { Xi) ^ 



(5.5) 



or equivalently, 

maximize Sum/i [Sum/j [ (DEIWj) (PPDAji) (Xi) ]] 
subject to 

Sum/i [ (Aki) (Xi) ] + ER (Bk) < Bk 
0 < Xi < 1 
i = 1...p 
j = 1 . .. d 
k = 1...t 



where , 



d = number of deficiencies. 
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DDEVi = ith program's deficiency derived effectiveness, 
DEI^vj = jth deficiency relative importance weight, and 
PPDAji = ith program's accomplishment of jth deficiency. 

Additional constraints concerning individual mission defi- 
ciencies which establish lower bounds on the deficiency's 
accomplishment level may be included in formulation 5.5. 
These lower bounds, however, introduce the possibility of no 
feasible linear programming solutions existing because the 
lower bounds are set too high. Adding a new variable, Yj, 
which represents the deviation from accomplishing the lower 
bound alleviates this possibility. With the variable Yj 
included the lower bound becomes a deficiency accomplishment 
goal. Including the additional deficiency accomplishment 
constraints the formulation becomes 

maximize Sum/i [ Sui/j [ (DF.IWj) (PPDAji) (Xi) ]] (5.6) 

subject to 

Sum/i [ (Aki) (Xi) ] + ER (Bk) < Bk 
Sum/i [ (PPDAji) (Xi) ] + (Yj) (DAj) > DAj 
0 < Xi < 1 
0 < Yi < 1 
i = 1 . .. p 
j = 1 . . . d 
k = 1...t 

where , 

DAj = jth deficiency's accomplishment level, and 

Yj = deviation from DAj achieved. 

The solution to formulation 5.6 will provide program 
selection, prioritization, and deficiency accomplishment 
information. Program selection criteria can be defined 
according to the funding level. If the ith RDT&E program's 
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is zero it follows that the ith EDISE 



fundirg level, 
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Xi, 

ct be selected under the constraints estab- 
near program. EDTSS programs that have low 
ding levels, Xi, serve to identify EDTSE 
y need to he reevaluated according to their 
plishment of mission deficiencies prior to 
ment . 

ze the RDTSE programs, two possibilities 
present MCDEC process and formulation 5.6. 
ioritization vectors are 



[ (EEEV) (X) ] 



(5.7) 



or. 



(X) [ (DDLW) (DDEV) + (SEIW) (S EV) ] = (X) (FPV) 



(5.8) 



where , 
SELR 
SE V 
DDlPi 
DDEV 
FPV 
X 



subjective evaluation list weight, 
subjective evaluation value, 
deficiency- derived list weight, 
deficiency- derived efficiency value, 
final program value, and 
program proportion funded. 



Eguations 5.7 and 5.E represe 
depending on the importance g 
listings weights represented 
by the Prioritization Working 
FPV in phase III of the MCEEC 
available, ranking programs by 
the current MCDEC pricritizati 
Formulation 5.6 also pr 
ciency accomplished according 
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iven to the vector SEV. The 
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Group (PRWG) to compute the 
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Sum/i [ (PFDAji) (Xi) 



(5.9) 



This information would be valuable to a decision maker for 
identifying deficiencies that are not fully accomplished. 
The additional variable, Yj, provides similar identification 
information concerning deficiencies not achieving their 
minimum accomplishment levels as set by the constant DAj. 

To further illcstrate the linear programming model 
formulations presented in this section a larger set of 
Eevelcpment Center-produced MCBEC process test data is used. 
The data involve 158 mission deficiencies and 74 EDTSE 
programs. The proxy budget data is obtained by using the 
same year ECT5E,Navy lOM. 

Table VIII shows the results of this larger data set 
when subjected to formulation 5.2. The table lists six 

columns. The first two columns from the left name and 
describe the budget conditions used. The next three list 
the number of RDT5E programs fully, partially, or not funded 
respectively. The last column provides the number of changes 
to the original program prioritization list when using 
formulation 5.2. The results depict the EDTSE program selec- 
tion produced by the linear programming model solution for 
maximizing the Final Program Value (FPV) subject to five 
progressively restrictive budget cases. As should be 
expected, fewer programs are funded fully as budget reduc- 
tions are imposed. However, if the FPV prioritization 
listing is used to select the funded programs, the optimal 
selection obtained through the linear program may not be 
produced. The right hand column of Table VIII provides the 
number of programs whcs priority position is lowered from 
the constrained case. This column contains the number of 
programs which lost their priority position because 
(FPVi) (Xi) < (FPVb) (Xi) , where i represents the program 
under consideration and b represents the program initially 
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ranked irmediatly belcw the ith program. These numbers are 
an indication of the the differences in selecting EDI&Z 
programs by a linear programming solution rather than the 
initial prioritized list produced in the liCDEC process. 



TABLE VIII 

Eesults of Formulation 5.2 with MCDEC Data 



Case 


EE 


Program Funding 
Full Part None 


Priority Changes 
FPV Parking 


A 


0. 00 


74 


0 


0 


0 


E 


0. 10 


72 


1 


1 


2 


C 


0. 15 


71 


1 


2 


2 


£ 


0.25 


69 


2 


3 


5 


E 


0.50 


61 


2 


11 


13 



The results of formulation 5.5 with the set of MCCZC 
process data is shown in Table IX. These results depict the 
linear program solution for maximizing the program’s 

Deficiency-Derived Efficiency Value (DDEV) when subjected to 
progressively restricted budget constraints. The ncticable 
solution difference between maximizing DDEV instead of FPV 
is the larger quantity of EDT5E programs dropped during the 
DDEV budget reduction conditions. The differences between 
the two formulation results is due to the lack of the 
Subjective Evaluation Value (SEV) in the DDEV solution. 
Assuming that the DDEV is a more direct and objective 
measure of F.DT5E program accomplishment of mission deficien- 
cies then in an actual application the partially funded and 
non-funded programs would be identified for further 
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€valuaticn prior to selection. The right hand ccluin of 
Table IX provides the number of programs whose priority 
position lowered when comparing a ranking of (Xi) (DDEVi) to 
a ranking according tc the vector DDEV only. 



TABLE IX 

Results of Ecrmulation 5.5 with MCDEC Data 



Case 


EE 


Erogiam Funding 
Full Part None 


Priority Changes 
DDEV Ranking 


A 


0. 00 


74 


0 


0 


0 


E 


0.10 


71 


1 


2 


3 


C 


0. 15 


69 


1 


4 


4 


r 


0.25 


60 


1 


13 


13 


E 


0. 50 


50 


2 


22 


23 



Ecrmulation 5.6 further restricted the maximizaticn of 
Sum/i [ (DDEVi) (Xi) ] by including constraints for each 

mission deficiency in addition to the budget constraints. 
Table X provides foriulation 5.6 results when using the same 
MCDEC data. In this illustration the jth Deficiency 
Accomplishment (DAj) value is set equal to a single quantity 
for all deficiencies. This value for DA represents a defi- 
ciency accomplishment goal which the model user would like 
the pcrtfclio of selected RDT&E programs to achieve. As 
previously stated/ this goal may not be feasible with 
currently considered programs. The variable Yj in formula- 
tion 5.6 provides assurance that a feasible solution will be 
obtained/ and it provides identification of the deficiencies 
that fail to meet the user's goal. If Yj is equal to zero 
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then the jth deficiercy goal, CAj, has been met. If the Yj 
value is one then no accomplishment of the gth deficiency is 
provided under the specified set of constraints. The results 
of four cases are sho*n in Table X. Four values of DA are 
used and each DA case is further subjected to five budget 
reduction conditions. For each case shown in Table X the 
budget reductions impose the same EDT&3 program selection 
portfolio as in formulation 5.5. The two priority change 
columns provide the number of programs that lower their 
priority standing as a result of the linear programming 
model reducing the funding proportion Xi to obtain an 
optimal solution. 

The deficiency accomplishment columns, of Table X, show 
the number of deficiencies that obtain the goal of EA in 
each case and the numbers of deficiencies partially and not 
accomplished. In the fully funded cases all RDT6E programs 
are selected as in the current MCDEC process and all defi- 
ciencies are at least partially accomplished. Case I-A shows 
that 74 deficiencies are greater than or equal to 100 
percent accomplished and 84 deficiencies are less than 100 
percent alleviated. Case IV-D shows that at a 25 percent 
budget reduction 87 deficiencies are accomplished at greater 
than cr equal to 85 percent while 63 deficiencies are less 
than £5 percent alleviated and eight deficiencies are not 
accomplished. For this data set, these results imply that 
even when all programs are fully funded many deficiencies 
are not accomplished at the set goal while others are using 
resources in excess cf that required to meet the goal. For 
the data used in this illustration 25 deficiencies were 
accomplished at 200 percent or greater and the largest 
cveracccmp lishment was 535 percent. Numbers representing 
deficiencies move ficm the DA column to partial and not 
accomplished columns as budget reductions were imposed that 
reduce funding for programs that support the specific 
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I 

TABLE X 

Results of EcrmulatioD 5.6 with MCDEC Data 



Case 


Pregram 
Fu nainq 




Priority 
Chan qes 


Deficiency 

Accomplishment 


E A EE 


Full 


Part 


None 


DDEV 


FPV 


DA i 


?art 


None 


Case I 


A 1 . C . CO 


74 


0 


0 


0 


0 


74 


84 


0 


3 . 10 


71 


1 


2 


3 


3 


74 


83 


• 1 


C . 15 


69 


1 


4 


4 


5 


73 


84 


1 


D .25 


60 


1 


1 3 


13 


1 4 


70 


83 


6 


E . 50 


50 


2 


22 


23 


24 


6 1 


87 


1 0 


Case II 


A .S5 .00 


74 


0 


0 


0 


0 


77 


81 


0 


3 . 10 


71 


1 


2 


3 


3 


77 


80 


1 


C . 15 


69 


1 


4 


4 


5 


76 


82 


1 


D .25 


60 


1 


1 3 


13 


14 


73 


77 


8 


E .50 


50 


2 


22 


23 


24 


6 4 


84 


1 0 


Case III 


A .9C .00 


74 


0 


0 


0 


0 


39 


69 


0 


B . 10 


71 


1 


2 


3 


3 


38 


69 


1 


C . 15 


69 


1 


4 


4 


5 


87 


70 


1 


D .25 


60 


1 


1 3 


13 


14 


33 


67 


8 


E . 50 


50 


2 


22 


23 


24 


76 


72 


1 0 


Case IV 


A .85.00 


74 


0 


0 


0 


0 


93 


65 


C 


3 . 10 


71 


1 


2 


3 


3 


92 


65 


1 


C . 15 


69 


1 


4 


4 


5 


91 


66 


1 


D . 25 


60 


1 


1 3 


13 


14 


87 


63 


8 


S . 50 


50 


2 


22 


23 


24 


30 


63 


1 C 



deficiency. In all DA goal cases, one mission deficiency was 
not acccnplished at any percent when a budget reduction of 
.10 cr .15 was imposed. When the more extreme budget reduc- 
tions were imposed mere programs were not funded and more 
deficiencies were not accomplished. The linear programming 
solution procedure ccncentrates resources into programs that 
accomplish deficiencies with the highest Deficiency Relative 
Importance Weight (DRIW) and does not prevent over accem- 
plishment in excess of 100 percent. This allows an optimal 
formulation to occur with a portfolio that does not select 
RDTSE programs that only accomplish deficiencies with a low 
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EHIW. As shown in case I-E/ 61 deficiencies were acccm- 
plished at cr greater than 100 percent while 10 deficiencies 
were not accomplished. At some point nonaccomplishment of 
the less important deficiencies will outweigh the benefit 
from cveraccomplishment of the more important deficiencies. 

The final linear programing formulation proposed here 
does net duplicate the MCDEC process in any obvious way but 
concentrates on maxiiiizing the quantity of all deficiencies 
accomplished. The following formulation uses the previously 
defined variable Yj as the deviation from the set deficiency 
accomplishment goal. The formulation for a goal of 100 
percent fellows as 

minimize Sum/j [Yj* (5.10) 

subject to 

Sum/i [ (Aki) (Xi) ] + ER (Bk) < Bk 
Sum/i [ (PPEA ji) (Xi) ] + Yj > 1.00 
0 < Xi < 1 
0 < Yi < 1 
i = 1 . . . p 

j = 1 . . . d 
k = 1 . .. t. 

Formulation 5.10 optimizes deficiency accomplishment by 
minimizing the deviation from full deficiency alleviation. 
Programs are restricted by budget constraints and are 
selected according tc the proportion of deficiency accom- 
plishment they impart to deficiencies still below the set 
goal. Table XI summarizes the results of Formulation 5.10 
with the MCEEC process data set. In general, fewer pregrams 
are required to provide the same mission deficiency accom- 
plishment than when all programs are fully funded. Net until 
an extreme budget reduction is imposed does the quantity of 
100 .ercent or grea^■sr accomplished deficiencies deer 
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By ccncentrating on the number of deficiencies accom- 
plished, fcrmulation 5.10 identifies programs which may not 
be reguired even when funding is available. The solution 
values of Y j provide an identification of the deficiencies 
that require more BETErZ program dedication to achieve full 
accomplishment. Table XI case A shows that for the same 
deficiency accomplishment results as case I-A of Table IX 
only 61 pregrams recuire full funding and five need only 
partial funding. Case I-A of Table IX allowed 7U fully 
funded programs. This reduction in selected programs repre- 
sents an IS. 7 percent total funding reduction over the five 
budgeted years and a 9.9 percent funding reduction in the 
most restrictive year. The FDT&E programs not funded in 
Table XI represent those which should be reevaluated as to 
their extent of deficiency accomplishment prior tc selec- 
tion. If these pregrams are not selected this may lead to 
the recovery of resources tc provide greater accomplishment 
cf the partially alleviated def iciencies. 

Unless the values of DDEV and FPV are used, formulation 
5.10 does not provide a means to completly prioritize a 
listing cf EDTSE pregrams. Formulation 5.10 does separate 
the programs into the three funding classifications of 
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and not funded. 



fully, fartially, and not funded. If a priori tinaticn is 
required, fully funded programs would logically form the 
highest priority and the not funded programs would fore the 
lowest priority classif ica tion set. The partially funnel 
programs would form the middle priority set of programs and 
could he further narked within this set according to tneir 
respective funding proportion Xi. 



C. EISCCSSION 

Two general approachs to the MC. nC prioritization 
process using linear programnii.g have bee., proposed. "...j 
first approach demonstrated in for mulati.^ii..^ b.'J, 5.2, S.-*, 

5.5, and 5,6 maximized the JICDEC process values concerning 
RDTGZ program relationships to mission deficiency accem- 
plishment. Each of these formulations have an unconstrained 
case where they duplicate the currently proposed XCLEC 
process. Formulation 5.10 represents a second method to 
approach the MCDEC process through concentraticn on the 
quantity of deficiencies accomplished and selecting programs 
as needed to obtain an accomplishment goal. These approaches 
and fcrmulations illustrate the feasibility and flexicility 
of the linear programming portfolio selection .mo id as 
applied to the MCDEC process. The results listed in Tarles I 
and XI show that for small budget reductions both aiproacnes 
select programs that provide similar quantities of d^^Iicien- 
c_es acce mplish ed tut formulation 5.10 provides i less 
costly alternative tc achieve these deficiencies. 

I'c accept the results of any linear programming proach 
a major assumption must be accepted. Linear progru;.ming 
assumes that the relationships of coefficients and variables 
used in' the formulation are linear or, at least, that linear 
functions satisfactorily approximate the actual relaticn- 
ships. Intuitively, and as presented in some of the 
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literature^ these interrelationships between 3015 E programs, 
resource expenditures, and benefits may not be linear but 
instead may be seme ether functional form. The linear func- 
tions, however, do provide an approximation to the actual 
situation which is u rderstandable. The proposed formulations 
are possibilities which can serve as program selection tools 
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ith program in the kth time 
on maintains that a program 
t less than one will yield the 
In actual applications the 
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coefficients must be accepted themselves as values capable 
cf retaining their benefit measurement properties during 
linear programming ccaputat ions. Stevens [Ref. 26] proposed 
the four measurement scales of nominal, ordinal, interval, 
and ratio which are helpful to discuss the properties of 
these coefficients. 

Briefly, nominal scale measurements represent numerical 
identification f cr an item cr event such as an auto license 
plate number and does not have any mathematical properties 
associated with it. Ordinal scale values measure rank 
crderir.g only such as the priority numbers of 1 through 74 
given tc the EET6E programs as a result of the KCEEC 
process. Interval and ratio scales maintain a comparative 

worth between values. The interval scale has an established 
interval unit and each specific value is measured by the 
number of these units that separate it from other measure- 
ment values above or below it cn the scale. A natural origin 
is net always established in an interval scale even if a 
2ero value exists. The zero values on interval scales are 
assigned to arbitrary points such as on the Celsius or 
Fahrenheit temperature scales. Ratio scales have a natural 
origin in addition tc the properties of the interval scales. 
Ratio scales are capable of proportional comparisons cf the 
numerical values such as stating a program which costs two 
millicn dollars is twice as expensive as a one million 
dollar program. To maintain the same measurement properties, 
ratio scales can withstand multiplicative transformations 
only. linear transformations can result in other ratio 
scales, however, the new values will be measures cf a 
different attribute than the first. 

In linear equations, values with at least interval scale 
properties are necessary. In linear programming, however, 
the coefficients require the additional properties of ratio 
scale measurements to withstand the ratio comparisons of 
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coefficients and right hand side values. In Chapter IV, 
FPDAji is defined as a straightforward estiaate of the ith 
progran's proportion of the jth deficiency accomplishment. 
If these estimates are accurate, the coefficient PPBAji can 
he acceptable as a measurement on a ratio scale. The BDZVi 
coefficient is a summation of individual proportional trans- 
formations cf each PPCAji based on the Deficiency relative 
Importance Keight for the jth deficiency (DEI'/ij) . The DDZVi 
can be accepted as a different ratio scale measurement by 
accepting as valid the multi-attribute utility analysis 
which is used to derive the DRIWj. The FPVi is a linear 
function of DDEVi and a Delphi process-based Subjective 
Evaluation Value (SEVi) . DDEVi and SEVi are both adjusted by 
the confidence factors of the Deficiency Derived List Weight 
(DDLW) and the ith program’s Subjective Evaluation List 
Weight (SElWi) , respectively. Although not specifically 
stated by Creed, [Ref. 25], all DDEVi in the sample KCDEC 
data set were adjusted by a common DDLW while each SEVi was 
adjusted by its specific SELWi. Accepting FPVi as a 
different ratio measurement of RDT5S program benefit 
requires accepting (SEVi) (SELWi) added to (DDEVi) (DDLW). 
Although both FPVi and DDEVi might be accepted as ratio 
scale measurements cf RDTSE pregram benefit, because of the 
functional change between them, they do not measure the same 
attributes cf the ith program. The vector (SEV) (SEIW) also 
does not represent the same qualities as the vector SEV 
because cf the evident change of SELWi according tc each- 
program. 

The KCDEC prioritization process proposed only the FPVi 
to represent the final relative worth of RDTSE programs in 
the accomplishment of mission deficiencies. If tuis value is 
acceptable then formulations 5.2 and 5.4 provide examples of 
linear programming portfolio selection models which will 
maintain an optimal program selection portfolio under budget 
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resource reductions. If the subjective evaluation addition 
included in the FPVi definition is not acceptable then the 
multi-attribute analysis of individual deficiency importance 
can be included by maximizing Sum/i [ (DDEVi) (Xi) ] as in 
formulations 5.5 and 5.6. These formulations allow program 
selection scluticns under budget reductions which may not be 
obtained through priority listings only. The last formula- 
tion approach, which does not directly represent the 
currently proposed HCEEC process, appears to be a favorable 
EDT6E program selection model as shown from the example data 
set. Formulation 5.10 provides identification of programs 
requiring reevaluaticn, deficiencies requiring dedicated 
programs, and a selection of programs that accomplish the 
maximum deficiency alleviation. It should be noted that in 
all the proposed linear program formulations the solutions 
are optimal computaticns of the benefit and coefficient data 
provided and can only be given the confidence inherent in 
the data themselves. 
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VI. COKIOSIONS 



A. EISUIIS 

This thesis has investigated the MCDEC Prioritization 
Process. .This process was produced by the Development 
Center, I^CDIC, to establish the relative importance of EDISE 
programs prior to Marine Corps input to the EDTSE,N PCM. An 
investigation of tie literature showed that the MCDEC 
process can be classified with similar benefit measurement 
models currently accepted by industry. The literature also 
identified ether proposed and accepted EDT5S program selec- 
tion models, classified as portfolio selection models, which 
showed possible advantages that could enhance t he proposed 
MCDEC process. Several MCDEC process linear programming 
model modifications were presented and showed the portfolio 
selection model advantages of maximizing the benefit func- 
tion when subjected to resource constraints. 

The feasibility cf representing the MCDEC process as a 
linear programming portfolio selection model was illus- 
trated. These i llustration s raised the possibility that the 
Final Program Value assigned in the process might not be the 
test measurement of ECT&E program mission deficiency accom- 
plishment. Another linear programming model approach to 
selecting the EDTSE programs, represented by the example 
data, was also illustrated and provided an economical alter- 
native for maximizing deficiency accomplishment. 

Currently a mathematical analysis support model is used 
as a tool in the Procurement Marine Corps PCM process tut 
not in the development of the Marine Corps portion of the 
EDTSE, Navy POM. According to the literature, industry 
accepts the use of subjective input with analytic tools for 
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EDTSE frcgram selecticn d[. plications. Ihe JICDEZ process as 
it stands or with the portfolio selection enhanceir.er.ts has a 
tase to assist as an analytic tool for the Marine Corps 
respcnsitilities in the RDTSE,N PCM process. 

E. EDEIEEE IH VESTIGAHONS 

Several areas of further study and research are evident 
from this investigation. The variety of program selection 
models proposed in the literature each have advantages and 
disadvantages which require further investigation as to 
their possible application to the Marine Corps RDI5E pregram 
selecticn process. Ihe data gathering and compiling tech- 
nigues of these additional models require study as well as a 
complete analysis of the data and process results from a 
complete trial MCDEC process run. The most difficult further 
research will be to estimate the model costs and perform a 
cost benefit analysis of implementing a RDTSE program selec- 
tion model process. 

Ihe literature review chapter presented model evaluation 
criteria ard limitations suggested for industrial PSD 
program selecticn models.- Further research is required 
into the applications of these or similar criteria and limi- 
tations in the context of Marine Corps applications specifi- 
cally. Ihe extent cf industrial model applications tc the 
Marine Corps will be limited because of the non-profit 
orientation of defense. However, the other portfolio selec- 
tion models reviewed in the literature may provide valuable 
alternatives to the presented MCDEC linear programming 
model . 

Subjective evaluation of RDT&E programs was seen 
accepted as a necessity in the literature and was seen 
proposed for use in the MCDEC process. The manipulation of 
the subjective data by the MCDEC process was discussed here 
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in ccntest cf measur €nent scale properties. Further investi- 
gation into the subjective values and their true measure- 
ments is needed. Saaty £Ref. 24] and Lindsay [Ref. 27] have 
proposed different methods of combining subjective judge- 
ments into Quantities that maintain measurement scale prop- 
erties. Further investigation into these or other subjective 
judgement methods may provide more acceptable alternatives 
than the Delphi and value manipulation methods currently 
proposed . 

Formulation 5.4 illustrated the possible resource allo- 
cation advantages from assigning a proportional pregram 
funding variable to each year in the planning horizon. Ihe 
benefit received from the programs, however, may also change 
over time. Further study is necessary to investigate the 
program benefit measurement values used in the portfolio 
selection models and how these values change over time. 
Additional variables and benefit coefficients necessary to 
describe each time period will increase the computer storage 
and ccmputational recuirements and this increase will also 
need investigation. 

The costs and benefits cf implementing tn e proposed 
process also requires f’urther study. The current process 
software was developed for a Tectronix 4054 minicomputer 
and is not compatible on other systems. Resources in terms 
cf manhours and money will be required to upgrade or repro- 
gram process software to ensure compatibility with mirccem- 
puter systems currently at HQMC and the Development Center. 
The presented linear programming computations utilized a 
Ketrcn, Inc. MFSIII Dataform package as run on an IBM 3033 
computer. Further investigation into microcomputer linear 
programing software is necessary and represents additional 
costs. If large linear programing packages are required to 
enhance the process additional mainframe usage will add 
costs. Gathering the subjective data from operational and 
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non-cperaticna 1 units during the MCDEC process survey anal- 
ysis represents manpcwer and TAD costs not currently 
required. The total monetary benefits are possibly the 
hardest to classify cr estimate. The value of valid informa- 
tion concerning new defense systems and programs as they 
affect the nissicn accomplishment of the i^arine Corps will 
need further research to quantify. 
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APPMDIX A 

MCDEC PEIORIIIZATION PROCESS 



liis appendix displays the organizational structure ior 
the Eevelcpment Center, Deputy Chief of Staff for 
Developmental Coordination, Prioritization Working Group,- 
and Chief cf Staff’s Prioritization Committee in Figures 
A,1, A. 2, A. 3, and A. 4 respectively. The remainder cf the 
appendix details the phases and tasks of the ^CDSC prioriti- 
zation process and lists the specific Development Center 
agency responsible for completing eacn step. The purposes 
for each following tasks are cuoted from Creed [Ref. 25]. 
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Figure A. 1 



Development Center Structure 




figure A. 2 



Developmental Coordination Structure. 




Figure A. 3 



Prioritization Working 



Group Structure. 
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Figure A. 4 



Prioritization Committee Structure. 



A. 



Phase I: Determine deficiencies and requirements, 

Task 1. Preparation of plans 



Dates: 1 January to 28 February 

Agency: Planning and Evaluation Branch 

Purpose: To provide an approved basis, via guidance 
and direction set forth in Marine Corps plans, for 
subsequent Mission Area Analyses, by updating 
and/or revision of basic Marine Corps planning 
dccuments - i.e., MMECP and MLBP. 



Task 2. Survey Analyses 



Dates: 1 January to 30 March 
Agency: Analysis Support Branch 

Purpose: To gather opinion input from FMF and 

other appropriate ccmmands/agencies regarding the 
ef f ective ress of ongoing projects in the ESC 
program and tc solicit new program ideas. 



Task 3. Mission Area Analyses 
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Dates: 1 March to 30 April 

Agency: Planning and Evaluation Branch 

Purpose: To provide the basis for justification of 

ESE projects via specified Marine Corps deficien- 
cies resulting from Mission Area Analysis. 



E. Phase II: Prioritize deficiencies and requirements, 

TasJc 1. Mission Area Analysis Results Briefings 



Dates: 1 May to 15 May 

Agency: Planning and Evaluation Branch 

Purpose; To provide the Prioritization Working 
Group (PRWG) with Mission Area Element (s) 
deficiencies-criented briefings in preparation for 
the PEHG MAA prioritizations (Phase II Task 2). 



Task 2. Survey Analysis Results Briefings 



Dates: 1 May to 15 May 
Agency: Analysis Support Branch 

Purpose: To provide the Prioritization Working 

Group (PRWG) with FMI/major command and agency 
derived input obtained during Phase I, concerning 
deficiencies. 



Task 3 



Design of the 



Deficiency decision tree 



Dates: 15 May to 15 July 

Agency: Prioritization Working Group (PRWG) 



Purpose: To analytically 

attribute utility analysis, 
ordering af all Mission Area 
and an associated relative 
each . 



derive, vis irulti- 
a consolidated rank- 
(element) deficiencies 
importance weight of 



C. Phase III: Prioritize FDD programs, 

Task 1. Program Project briefings 
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Dates: 15 Jul}' to 15 August 

Agency: Division DFO’s (scheduled by Dperaticr.s 

Erancn) 

Purpose: To provide preparatory inforiEaticE to th 

Pricritizaticn Working Group and Chief of Staff* 
Ccmniittee £ cr consideration in subsequent priori ti 
zaticn deliberations. 



Task 2. Survey Analysis Results Briefing 



Dates: 15 July to 15 August 
Agency: Analysis Support Branch 

Purpose: To provide preparatory information 
derived from IHF/majcr commands or agencies to the 
Prioritization Working Group and Chief of Staff’s 
Committee for consideration in subsequent prioriti- 
zation deliberations. 



Task 3. Subjective Prioritization of ESD Program 
Projects 



Dates: 15 August to 1 October 
Agency: Analysis Support Branch 

Purpose: To obtain a subjective prioritization of 

ESD Projects with associated weighted values of 
each reflecting relative worth. 

Task 4. Deficiency- Derived Prioritization of PSD 
Pro jec ts 



Dates: 15 August to 1 October 

Agency: Prioritization Working Group (PRWG) 

Purpose: To obtain a prioritization of RSD program 

projects ranked and weighted according to that 
project's capability to overcome Mission Area 
Element def iciencies. 



Task 5. Final Prioritization of RSD projects 

Dates: 1 October to 15 October 

Agency: Prioritization Working Group (PRWG) 
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Purpose: To obtain a final prioritization of ESC 
program projects from the subjective-derived list 
and deficiency-derived list comoined. 



Task 6. Director Development Center, Decision Brief for 
Action 



Dates; 15 October to 31 October 
Agency: As Directed 

Purpose: To present the recommended RBD program 

project priority for decision. 



D. 

EOil. 



Phase IV: Complete MCCEC R & 

Task 1. Application of funding 



D pro grams 1 ist 



profiles 



in HQMC 



Dates: 1 November to 31 December 
Agency: Operations Branch 

Purpose: To determine recommended funding profiles 

to reflect the desired prioritization of H6C 
Program Projects. 
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